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(57) Abstract: A temporal recorsive filler unit (100,200,300,400) fornoisc filtering of a series of rrrpur images resulting in a scries 
of output images comprises: means (102) for determining a vaine of a weighing factor, on hasis of a difference between a first value 
of a first pixel of an input image and a second value of a second pixel of a Erst output image; and an adding (104) unit for calculating 
a third value of a third pixel of a second output image by adding of a first product of the value of the weighing factor and the first/ 
value of die first pixel, to a second product of a complement of the value of the weighing factor and the second value of the second 
pixel. The value (508) of the weighing factor is higher below a predetermined threshold (506) than above the threshold (506). 
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NOISE FILTERING IN IMAGES 



The invention relates to a temporal recursive filter unit for noise filtering of a 
series of input images resulting in a series of output images, comprising: 

- means for determining a value of a weighing factor, on basis of a difference 
between a first value of a first pixel of an input image of the series of input images and a 
second value of a second pixel of a first output image of the series of output images; and 

- an adding unit for calculating a third value of a third pixel of a second output 
image of the series of output images by adding of a first product of the value of the weighing 
factor and the first value of the first pixel, to a second product of a complement of the value 
of the weighing factor and the second value of the second pixel. 

The invention further relates to method of noise filtering of a series of input 
images resulting in a series of output images, comprising: 

- a weighing factor determination step of determining a value of a weighing 
factor, on basis of a difference between a first value of a first pixel of an input image of the 
series of input images and a second value of a second pixel of a first output image of the 
series of output images; and 

- an adding step of calculating a third value of a third pixel of a second output 
image of the series of output images by adding of a first product of the value of the weighing 
factor and the first value of the first pixel, to a second product of a complement of the value 
of the weighing factor and the second value of the second pixel. 

The invention further relates to an image processing apparatus comprising: 

- receiving means for receiving a series of input images; and 

- such a temporal recursive filter unit for noise filtering of the series of input 
images resulting in a series of output images. 

A unit of the kind described in the opening paragraph is known from US 
6,1 15,502. In that patent is described that a fresh input signal and a previously filtered signal 
are combined in the proportion k: (1-k), where k depends on a local amount of motion, hi this, 
maimer, it is attempted to avoid smear obtained by averaging signals from mutually differing 
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temporal instants in the presence of motion, while the noise filtering is folly active in the 
absence of motion. The variable k can be seen as a factor determining how much fresh input 
directly influence the filter output The variable k is determined with a so-called motion 
detector. The variable is based on luminance differences between pixels of input images and 
5 output images. It is assumed that the luminance difference between input and output is a 
measure for the amount of motion. The value of variable k as function of luminance 
difference is monotonous: the higher the luminance differences between pixels the lower the 
value of variable k. Typically the value of k ranges from zero to one. A small difference is 
typically considered noise, and thus the k value will be close to zero, resulting in strong 
1 0 filtering. A large difference between input and output typically identifies motion in the scene 
and results in a higher value of k, thus preserving as much image detail as possible. 

In a fixed point arithmetic implementation of a unit of the kind described in 
the opening paragraph, the internal calculations require a higher precision, i.e. word size than 
which is required to represent the input and output images. So, prior to the output of the unit, 
15 the accuracy of the signal has to be reduced. In a straightforward implementation the internal 
signal is rounded and the unused bits truncated. For example a 12 bits intermediate value is 
rounded to 8 bits. First the value 0.5 in fixed point 4 bits notation is added. Then the 4 least 
significant bits are removed by truncation. Such a filter unit, based on fixed point arithmetic, 
suffers from a known artifact, caused by the recursive nature of the filter unit. The value of 
20 an output pixel provided by the recursive filter unit will generally not reach the required 
value after a sudden change in the input signal. This artifact is known as "long term dirty 
window effect". For example, when the input signal changes from a picture to black, a vague 
remainder image of the original input signal is left on the display. 



25 

It is an object of the invention to provide a filter unit of the kind described in 
the opening paragraph in which the above described artifact hardly occurs. The object of the 
invention is achieved in that the means for determining the value of the weighing factor is 
arranged to provide the value of the weighing factor, which is higher than a further value of 
30 the weighing factor if the difference between the first value and the second value is below a 
predetermined threshold, with the further value belonging to a further difference of further 
values of further pixels, with the further difference being above- the predetermined threshold. 
Instead of applying a low value of the weighing factor in the case of a small difference , . 
between the pixels a relatively high value is applied. E.g; if the value of the weighing factor 
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ranges from [0,1] then the value of the weighing factor is set to 0.5 if the difference between 
the pixel values is below a predetermined threshold. This is not obvious, because it is 
assumed that a small difference between pixel values means no or hardly any movement and 
hence much filtering should be applied. Or in other words, the value of the new output pixels 
5 is primarily determined by the value of the previous output pixel and hardly on the input 
pixel. However according to the invention the amount of filtering should be low in the case 
of a difference between the value of the output pixel and the value of the input pixel, which is 
below a predetermined threshold. By applying less filtering, the influence of values of the 
input pixels on the values of the output pixels increases and hence the values of the output 
10 pixels converge to the required value. 

In an embodiment of the temporal recursive filter unit according to the 
invention, the predetermined threshold depends on calculation accuracy of the temporal 
recursive filter unit. Typically filter units are implemented by means of fixed point 
arithmetic. Above it is described that truncation is required to convert pixels represented by 
15 N number of bits to M number of bits. Before truncation, an offset is added. Typically this 
offset is equal to 0.5 times the value of the least significant bit in the representation with M 
bits. The predetermined threshold is related to the size of the offset being used. In other 
words the predetermined threshold is related to'the number of bits being used to represent the 
images. See Fig. 1 and Fig. 2 for examples. 
20 An embodiment of the temporal recursive filter unit according to the invention 

comprises an error diffusion unit for diffusing truncation errors which are made by 
conversion of an intermediate image into the second output image. Error diffusion is another 
approach to deal with the "long term dirty window effect". By applying the invention in a 
temporal recursive filter unit with an error diffusion unit, the convergence to the required 
25 output value is improved. 

An embodiment of the temporal recursive filter unit according to the invention 
comprises a motion compensation unit for matching the first pixel with the second pixel. It is 
advantageous to apply motion estimation in combination with motion compensation in the 
temporal recursive filter unit according to the invention. By means of that corresponding. 
30 pixels of successive images can be mixed. 

Modifications of the temporal recursive filter unit and variations thereof may 
correspond to modifications and variations thereof of the method described and of the image 
processing apparatus described. 
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These and other aspects of the temporal recursive filter unit, of the method and 
of the image processing apparatus according to the invention will become apparent from and 
will be elucidated with respect to the implementations and embodiments described 
5 hereinafter and with reference to the accompanying drawings, wherein: 

Fig. 1 schematically shows an embodiment of the temporal recursive filter unit 
according to the invention; 

Fig. 2 schematically shows an embodiment of the temporal recursive filter unit 
comprising an error diffusion unit; 
10 Fig. 3 schematically shows an embodiment of the temporal recursive filter unit 

comprising a motion compensation unit; 

Fig. 4 schematically shows an alternative implementation of an embodiment of 
the temporal recursive filter unit; 

Fig. 5A schematically shows the value of the weighing factor as function of 
1 5 the difference between pixels according to the prior art; 

Fig. 5B schematically shows the value of the weighing factor as function of 
the difference between pixels according to the invention; and 

Fig. 6 schematically shows an embodiment of the image processing apparatus 
according to the invention. 
20 Corresponding reference numerals have the same meaning in all of the Figs. 



Fig. 1 schematically shows an embodiment of the temporal recursive filter unit 
100 according to the invention. The temporal recursive filter unit 100 comprises: 

25 - means 102 for determining a value of a weighing factor a(x, n) for a first 

value C(x, n) of a first pixel of an input image of a series of input images and a second value 
P(x, « - 1) of a second pixel of a first output image of a series of output images, on basis of a 
difference between the first value and the second value; 

- an adding unit 1 04 for calculating a third value P{x, ri) of a third pixel of a 

30 second output image of the series of output images by adding of a first product of the value of 
the weighing factor a(x 7 n) and the first value C{x,n) of the first pixel, to a second product of 
a complement 1 - a(x, n) . of the value of the, weighing factor a(x, n) and the second value 
P(x, n-V)of the second pixel; and 
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- a memory unit 106 for storage of the first output image. This is a required for 
introducing a delay. 

The index n denotes an image number and the vector x corresponds to the 
coordinates of a pixel. At the input connector 108 the series of input images is provided. The 
5 temporal recursive filter unit 100 provides the series of output images at its output connector 
1 1 0. The means 1 02 for determining the value of the weighing factor a(x> n) is arranged to 
determine the value based on comparing pixel values of input and output images. This can be 
by taking into account the luminance values of only two pixels, i.e. one pixel from the current 
input image and one pixel from the previously filtered output image. However preferably 
10 several p ixels in the neighborhood of the pixels are taken into account. Jn US 6,1 15,502 an 
example of the calculation of the weighin g factor k is specified. This can be rewritten to 
Equation 1: 

with C(x, n) the value of the input pixel at position x for image n and P(x, n - 1) the value 
15 of the output pixel at position x for image n~\ and where N x and N 2 are neighborhoods 
around the current pixel. LUT means a look-up-table function. 

The transfer function of the temporal recursive filter unit 100 can be described 
with Equation 2: 

P(x, n) = a(x, n)C(x, n) + (1 - a(x 7 n))P(x> n - 1) (2) 
20 By means of an example it will be explained how the temporal recursive filter 

unit according to the invention works. The example shows how the value P(x,n) of an output 
pixel of a recursive filter changes when the value of the input pixel C(x, n) changes from 
CQcft) = 100 to C(Xyl) s= 10 . The example comprises 3 parts: 

- In Table 1 it will be demonstrated that the value of the output pixel P{&n) 
25 converges to the required value, in the case of a filter unit which is not restricted by a limited 

word size. That means that no truncation is applied. 

- In Table 2 it will be demonstrated that the value of the output pixel 

P(x, n) does not converge to the required value, in the case of a filter unit in which truncation 
is applied. 

30 - In Table 3 it will be demonstrated that the value of the output pixel 

P(x, n) converges to the. required value, in the case of a filter unit in which truncation is . .. 
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applied and in which the invention is applied: an embodiment of a temporal recursive filter 
unit according to the invention. 

Table 1 : Step response in a filter unit with maximum accuracy 
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5 

The values of P{x,n) in Table 1 are calculated by means of Equation 3: 
P(in) = (a(i,/2)C(x,;t) + (16-a(^«))P(jc,«--l))/16 (3) 
The value of the weighing factor a(x 9 n) ranges from [1,16] and is set to 14 for n = 1 and is 
set to 1 for n = 0,2,3,4,... The value of the weighing factorage, n) depends on the difference 
1 0 between Pfe n - 1) and C{x, n) . In Table 1 it can be seen that the value of P(x, n) converges 
very slowly to the required value 10: forw = 96 the value of P(x,ri) = 10.02446 . 
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Table 2: Step response in a filter unit according to the prior art. 
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The values of P(x,n) in Table 2 are calculated by means of Equation 4: 
5 P(x, n) = truncatetiflix, n)C(x, n) + (16 - w - 1) + 8) / 16) (4) 

This corresponds with a fixed point representation where the input and output data is 
represented with 8 bits. Before truncation an offset of 8/16 is added. In Table 2 it can be seen 
that the required value 10 is not reached. Because of the truncation the value of P(&n) does 
not become lower than 18. 
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Table 3: Step response in a filter unit according to the invention 
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The values of P(x> n) in Table 3 are calculated by means of Equation 4. The 
5 difference with Table 2 is that now the value of the weighing factor n) is set to 9 for j 
n = 0,5,6,7,.. The value of the weighing factorage, n) depends on the difference between 
P(x, n - 1) and C(x, n) . In Table 3 it can be seen that the required value 1 0 is reached. This is j 
because the value of the weighing factor a{x, n) is set to a high value for small differences j 
between P(x, n - 1) and C(x, n) . \ 
1 0 Fig. 2 schematically shows an embodiment of the temporal recursive filter unit 

200 comprising an error diffusion unit 202. In stead of a fixed rounding by means of adding a 
constant offset of 0.5 the error diffusion unit 202 of this temporal recursive filter unit 200 
according to the invention preserves the truncation error made for a pixel and uses this as a ; 
variable "offset" for a succeeding pixel. Note that a spatial error diffusion can be applied. A ■; 
1 5 standard truncation works as specified in Equation 5 : . 

Outpuiix) = truncate{Input(i) -f 0.5) (5) 
with index i . The error diffusion unit 202 works as specified in Equation 6: 

Output(i) = tnmcate{fnput{i) + rest) (6) 

I 

with,. ; 
20. rest = Inpid(i-l)-tnaicate{Input{i--l) (7) 

< 
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Substitution of Equation 7 into Equation 6 yields: 

Output{i) = tnmcate({Input(i) + (Jnpnt(i - 1) - truncate(Input(i - 1)) (8) 
Table 4 gives an example of a standard truncation with a fixed offset of 0.5 according to 
Equation 5 and Table 5 gives an example of a truncation based on error diffusion according 
to Equation 8. 

Table 4: Standard truncation 
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Table 5: Truncation based on error diffusion 
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By means of an example it will be explained how the temporal recursive filter 
unit 200 according to the invention works. The example shows how the value P(x,n) of an 
output pixel of a recursive filter changes when the value of the input pixel C{x, n) changes 
from C(x,0) = 100 to C(x,l) = 10. The example comprises 2 parts: 

- In Table 6 it will be demonstrated that the value of the output pixel 

P(x,n) converges to the required value very slowly in the case of a filter unit according to the 
prior art in which error diffusion is applied. 

- In Table 6 it will be demonstrated that the value of the output pixel 
P(x^n) converges much faster to the required value, in the case of a filter unit according to 
the invention in which error difiusion is applied:: 
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Table 6: Step response in a filter unit according to the prior art with an error diffusion unit 
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The values of P(x, n) in Table 6 are calculated by means of Equation 9: 
P(x,n) = t7imcate((a(x, n)C(x, n) + (16 - a(x,n))P{x, n-l) + rest) i 16) (9) 
with rest ranging from [0,15] and being calculated as specified in Equation 7. The value of 
the weighing factor a (x,n) ranges from [1,16] and is set to 14 for n = 1 and is set to 1 for 
n = 0,2,3,4,...The value of the weighing factora(jc, n) depends on the difference between 
P(x, n - 1) and C(x, n) . The output pixel P(x, n) converges to the required value very slowly. 
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Table 7: Step response in a filter unit according to the invention with an error diffusion unit 
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The values of P(x, n) in Table 7 are calculated by means of Equation 9, with 
5 rest ranging from [0,15] and being calculated as specified in Equation 7. The value of the 
weighing factor a(x, granges from [1,16] and is set to 14 for n = 1 and is set to 1 for 
n = 2,3,. .,6 and set to 8 for n = 0,7,8,9,.. The value of the weighing factora(x,/z) depends, on 
the difference between P(x, n - 1) and C(x, n) . The output pixel P(x> n) converges to the 

required value much faster. ...... . 

10 Fig. 3 schematically shows an embodiment of the temporal recursive filter unit 

300 comprising a motion compensation unit 302. Because of motion in the scene being 
captured, pixels from successive images with mutually equal coordinates will not correspond 
to the same portions of objects in the scene. In order to match corresponding pixels motion 
estimation is required resulting in a motion .vector field comprising an arrangementof motion 

15 vectors. The motion compensation unit 302 is arranged to match corresponding pixels based 
on the estimated motion.vectors. 
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Fig. 4 schematically shows an alternative implementation of an embodiment of 
the temporal recursive filter unit 400 according to the invention. The behavior of the 
temporal recursive filter unit 400 corresponds with the temporal recursive filter unit 100 
described in connection with Fig. I. The advantage of this implementation is that only one 
5 multiplication unit 406 is required. But note that the arrangement of the subtraction unit 404, 
the multiplication unit 406 and the addition unit 408 results in an addition of a first product of 
the value of the weighing factor a(x, n) and the first value C(x, n) of the first pixel, to a 
second product of a complement 1 - a(x, n) of the value of the weighing factor a (x, n) and 
the second value P(x, /* - 1) of the second pixel. 
1 0 The size of the memory unit 1 06 for storage of an output image, in any of the 

temporal recursive filter units 100, 200, 300 or 400, might be such that an output image can 
be stored with the same nimnber of bits per pixel as being used to represent the output image 
provided at the output connector 110. Optionally embedded compression is applied to reduce 
the size of the memory unit. This is not shown in any of the Figs. 1-4. Especially in the case 
15 of lossy compression it is advantageous to apply the invention. 

Fig. 5A schematically shows the value of the weighing factor a as function of 
the difference between pixels according to the prior art. The x-axis 502 corresponds to a 
measure based on the difference between the value of a pixel of the input image and the value 
of a pixel of the output image. The y-axis 504 corresponds to the weighing factor a . The 
20 function is monotonously increasing. In other prior art, e.g. US 5,1 19,195 also curves 
showing the value of variable k as function of motion are provided. These curves have a 
similar shape: not-decreasing. The higher the motion, i.e. the difference between input and 
output images, the higher the value of the variable k. 

Fig. 5B schematically shows the value of the weighing factor a as function of 
25 the difference between pixels according to the invention. The x-axis 502 corresponds to a 

measure based on the difference between the value of a pixel of the input image and the value 
of a pixel of the output image. The y-axis 504 corresponds to the weighing factor a . Two 
sub-curves are depicted: one below the predetermined threshold 506 and one above the 
predetermined threshold 506. Below the predetermined threshold 506 the value of the 
30 weighing factor- a is relatively high' compared with values belonging to the sut>curve above 
the predetermined threshold. Above the predetermined threshold 506 the;value of the 
weighing factor a increases for larger differences between the values of the pixels. Hence, a 
first value 508 corresponding to- a- difference being lower than tfae.predetomined threshold is 
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higher than a second value 510 corresponding to a difference being higher than the j 
predetermined threshold. 

The value of the weighing factor a below the predetermined threshold is equal j 
to 0.5. This is just an example value. Besides that it is possible that there are multiple values 
5 below the predetermined threshold, e.g. a function of the weighing factor a with a staircase 
shape. 

Fig. 6 schematically shows an embodiment of the image processing apparatus j 
600 according to the invention, comprising: 

- receiving means 602 for receiving a series of input images. The received 

10 signal may be a broadcast signal received via an antenna or cable but may also be a signal ; 
from a storage device like a VCR (Video Cassette Recorder) or Digital Versatile Disk j 
(DVD). The signal is provided at the input connector 608. j 

- a temporal recursive filter unit 604 for noise filtering of the series of input j 
images resulting in a series of output images as described in connection with any of the Figs. 

15 1-4. j 

- display means 606 for displaying the series of output images. ; 
The image processing apparatus 600 might be a TV. 

It should be noted that the above-mentioned embodiments illustrate rather than : 
limit the invention and that those skilled in the art will be able to design alternative 

20 embodiments without departing from the scope of the appended claims. In the claims, any 

reference signs placed between parentheses shall not be constructed as limiting the claim. :. 

The word 'comprising' does not exclude the presence of elements or steps not listed in a 

claim. The word "a" or "an" preceding an element does not exclude the presence of a 

plurality of such elements. The invention can be implemented by means of hardware ; 

25 comprising several distinct elements and by means of a suitable programmed computer. In 
the unit claims enumerating several means, several of these means can be embodied by one 
and the same item of hardware. 

■ 
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CLAIMS: 



1. A temporal recursive filter unit (100,200,300,400) for noise filtering of a 

series of input images resulting in a series of output images, comprising 

- means (102) for determining a value (508) of a weighing factor, on basis of a 
difference between a first value of a first pixel of an input image of the series of input images 

5 and a second value of a second pixel of a first output image of the series of output images; 
and 

- an adding (104) unit for calculating a third value of a third pixel of a second 
output image of the series of output images by adding a first product of the value of the 
weighing factor and the first value of the first pixel, and a second product of a complement of 

10 the value of the weighing factor and the second value of the second pixel, characterized in 
that the means (102) for determining the value (508) of the weighing factor is arranged to 
provide the value (508) of the weighing factor, which is higher than a further value (510) of 
the weighing factor if the difference between the first value and the second value is below a 
predetermined threshold, with the further value belonging to a further difference of further 

15 values of further pixels, with the further difference being above the predetermined threshold. 

2. A temporal recursive filter unit (100,200,300,400) as claimed in Claim 1, 
characterized in that the predetermined threshold depends on calculation accuracy of the 
temporal recursive filter unit (100,200,300,400). 

20 

3. A temporal recursive filter unit (200,300) as claimed in Claim I, characterized 
in comprising an error diffusion unit for diffusing truncation errors which are made by 
conversion of an intermediate image into the second output image. 

25 4. A temporal recursive filter unit (300) as claimed in Claim 1 , characterized in 

comprising a motion compensation unit for matching the first pixel with the second pixel- 



s' A method of noise filtering of a series of input images resulting in a series of 

output images, comprising 
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- a weighing factor determination step of determining a value of a weighing 
factor, on basis of a difference between a first value of a first pixel of an input image of the 
series of input images and a second value of a second pixel of a first output image of the 
series of output images; and 

5 - an adding step of calculating a third value of a third pixel of a second output 

image of the series of output images by adding of a first product of the value of the weighing 
factor and the first value of the first pixel, to a second product of a complement of the value 
of the weighing factor and the second value of the second pixel, characterized in that in the 
wei ghin g factor deter minat ion step the value (508) of the weighing factor is determined, 
10 which is higher than a further value (510) of the weighing factor if the difference between the 
first value and the second value is below a predetermined threshold, with the further value 
belonging to a further difference of further values of further pixels, with the further 
difference being above the predetermined threshold. 

15 6. An image processing apparatus (600) comprising: 

- receiving means (602) for receiving a series of input images; 

- a temporal recursive filter unit (100,200,300,400) for noise filtering of the 
series of input images resulting in a series of output images, comprising 

* means (102) for determining a value of a weighing, on basis of a difference 
20 between a first value of a first pixel of an input image of the series of input images and a 

second value of a second pixel of a first output image of the series of output images; and 

* an adding unit (104) for calculating a third value of a third pixel of a second 
output image of the series of output images by adding of a first product of the value of the 
weighing factor and the first value of the first pixel, to a second product of a complement of 

25 the value of the weighing factor and the second value of the second pixel. 

Characterized in that the means (102) for determining the value (508) of the. 

weighing factor is arranged to provide the value (508) of the weighing factor, which is higher 

than a further value (510) of the weighing factor if the difference between the first value and . 

the second value is below a predetermined threshold, with the further value belonging to a 
3 0 further difference of further values of further pixels, with the further difference being above 

the predetermined threshold. 



7. An image processing apparatus (600) as claimed in claim 6, characterized in 

further comprising display means (606) for displaying the series of output images. 
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8. An image processing apparatus as claimed in claim 7, characterized in that it is 

a TV. 
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